Body contour key point detection methods, apparatuses, and devices

ABSTRACT

Body contour key point detection methods, image processing methods, neural network training methods, apparatuses, electronic devices, computer-readable storage media, and computer programs include: obtaining an image feature of an image block including a body; obtaining a body contour key point prediction result of the body by means of a first neural network according to the image feature; and obtaining a body contour key point in the image block according to the body contour key point prediction result; where the body contour key point is used for representing an outer contour of the body.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation application of International Patent ApplicationNo. PCT/CN2019/070725 filed on Jan. 8, 2019, which claims priority tothe Chinese Patent Application No. CN 201810054352.3 filed on Jan. 19,2018. The disclosures of these applications are incorporated herein byreference in their entirety.

TECHNICAL FIELD

The present disclosure relates to computer vision technologies, and inparticular, to body contour key point detection methods, body contourkey point detection apparatuses, electronic devices, computer-readablestorage media, and computer programs.

BACKGROUND

At present, body key points generally refer to body skeleton key points,and the body key points are mainly used for constructing a body skeletonmodel. Applications such as body action identification or man-machineinteraction can be implemented according to the constructed bodyskeleton model.

How to obtain more semantic information from an image including a bodyso that the image may represent richer body structure semantics, therebyproviding richer information for the applications is a noteworthytechnical problem.

SUMMARY

Embodiments of the present disclosure provide a technical solution ofbody contour key point detection.

According to an aspect of embodiments of the present disclosure, a bodycontour key point detection method is provided. The method includes:obtaining an image feature of an image block including a body;obtaining, by means of a first neural network, a body contour key pointprediction result of the body according to the image feature; andobtaining, according to the body contour key point prediction result, abody contour key point in the image block; where the body contour keypoint is used for representing an outer contour of the body.

According to another aspect of the present disclosure, an imageprocessing method is provided. The method includes: detecting a bodycontour key point of an image block including a body, where the bodycontour key point is used for representing an outer contour of the body;and performing AR effect rendering processing on the image blockaccording to the body contour key point.

According to another aspect of the present disclosure, a neural networktraining method is provided. The method includes: obtaining an imagefeature of a sample image block including a body; obtaining a bodycontour key point prediction result of the body by means of a firstneural network to be trained according to the image feature; and using adifference between the body contour key point prediction result and thebody contour key point marking information as guidance information toperform supervised learning on the first neural network to be trained.

According to another aspect of the present disclosure, a body contourkey point detection apparatus is provided. The apparatus includes: animage feature obtaining module, configured to obtain an image feature ofan image block including a body; a prediction result obtaining module,configured to obtain a body contour key point prediction result of thebody by means of a first neural network according to the image feature;and a contour key point obtaining module, configured to obtain a bodycontour key point in the image block according to the body contour keypoint prediction result; where the body contour key point is used forrepresenting an outer contour of the body.

According to another aspect of the present disclosure, an imageprocessing apparatus is provided. The apparatus includes: a body contourkey point detection apparatus, configured to detect a body contour keypoint of an image block including a body, where the body contour keypoint is used for representing an outer contour of the body; and arendering module, configured to perform AR effect rendering processingon the image block according to the body contour key point.

According to another aspect of the present disclosure, a neural networktraining apparatus is provided. The apparatus includes: a first module,configured to obtain an image feature of a sample image block includinga body; a second module, configured to obtain a body contour key pointprediction result of the body by means of a first neural network to betrained according to the image feature; and a third module, configuredto use a difference between the body contour key point prediction resultand the body contour key point marking information as guidanceinformation to perform supervised learning on the first neural networkto be trained.

According to another aspect of the present disclosure, an electronicdevice is provided. The electronic device includes a processor; and amemory for storing instructions executable by the processor; whereinexecution of the instructions by the processor causes the processor toperform any method above when the computer program is executed.

According to another aspect of the present disclosure, a non-transitorycomputer-readable storage medium is provided. A computer program isconfigured to store computer-readable instructions, wherein execution ofthe instructions by the processor causes the processor to perform anymethod above.

According to another aspect of the present disclosure, a computerprogram is provided. The computer program includes computerinstructions, where when the computer instructions are operated in aprocessor of a device, any method above is implemented.

Based on the body contour key point detection method, body contour keypoint detection apparatus, image processing method, image processingapparatus, neural network training method, neural network trainingapparatus, electronic device, computer-readable storage medium, andcomputer program provided in the present disclosure, in the presentdisclosure, the first neural network is used for obtaining the bodycontour key point, thereby facilitating rapid and accurate precisesketch of the body outer contour in the image, so as to facilitateobtaining of richer body semantics from the image including the body, sothat body semantic information can be provided to a correspondingapplication, for example, the body contour key point extracted from thepresent disclosure may be used for applications such as stickerprocessing, body action analysis, and body morphology analysis.

By means of the accompanying drawings and embodiments, the technicalsolutions of the present disclosure are further described in details.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings constituting a part of the specification describeembodiments of the present disclosure, and are used for explaining theprinciples of the present disclosure in combination of the description.

With reference to the drawings, according to the detailed description,the present disclosure can be understood more clearly, where

FIG. 1 illustrates a flow chart of an embodiment of a body contour keypoint detection method of the present disclosure;

FIG. 2 illustrates a schematic diagram of an embodiment of a bodycontour key point of the present disclosure;

FIG. 3 illustrates a flow chart of an embodiment of a neural networktraining method of the present disclosure;

FIG. 4 illustrates a flow chart of another embodiment of the neuralnetwork training method of the present disclosure;

FIG. 5 illustrates a flow chart of an embodiment of a body contour keypoint marking information generating method of an image sample of thepresent disclosure;

FIG. 6 illustrates a flow chart of an embodiment of an image processingmethod of the present disclosure;

FIG. 7 illustrates a flow chart of an embodiment of sticker processingperformed on an image to be processed of the present disclosure;

FIG. 8 illustrates a structural schematic diagram of an embodiment of abody contour key point detection apparatus of the present disclosure;

FIG. 9 illustrates a structural schematic diagram of an embodiment of aneural network training apparatus of the present disclosure;

FIG. 10 illustrates a structural schematic diagram of an embodiment ofan image processing apparatus of the present disclosure; and

FIG. 11 illustrates a block diagram of an exemplary device forimplementing an embodiment of the present disclosure.

DETAILED DESCRIPTION

Each exemplary embodiment of the present disclosure is described indetail with reference to the accompany drawings now. It should be notedthat: unless otherwise stated specifically, relative arrangement of thecomponents and operations, the numerical expressions, and the values setforth in the embodiments are not intended to limit the scope of thepresent disclosure.

In addition, it should be understood that, for ease of description, thesize of each part shown in the accompanying drawings is not drawn inactual proportion.

The following descriptions of at least one exemplary embodiment aremerely illustrative actually, and are not intended to limit the presentdisclosure and the applications or uses thereof.

Technologies, methods and devices known to a person of ordinary skill inthe related art may not be discussed in detail, but such technologies,methods and devices should be considered as a part of the specificationin appropriate situations.

It should be noted that similar reference numerals and letters in thefollowing accompanying drawings represent similar items. Therefore, oncean item is defined in an accompanying drawing, the item does not need tobe further discussed in the subsequent accompanying drawings.

The embodiments of the present disclosure may be applied to electronicdevices such as terminal devices, computer systems, servers, which mayoperate with numerous other general-purpose or special-purpose computingsystem environments or configurations. Examples of well-known computingsystems, environments, and/or configurations suitable for use togetherwith the computer systems/servers include, but are not limited to,personal computer systems, server computer systems, thin clients, thickclients, handheld or laptop devices, microprocessor-based systems, settop boxes, programmable consumer electronics, network personalcomputers, small computer systems, large computer systems, distributedcloud computing environments that include any one of the foregoingsystems.

The electronic devices such as terminal devices, computer systems, andservers may be described in the general context of computer systemexecutable instructions (for example, program modules) executed by thecomputer system. Generally, the program modules may include routines,programs, target programs, components, logics, and data structures, toexecute specific tasks or implement specific abstract data types. Thecomputer systems/servers may be practiced in the distributed cloudcomputing environments in which tasks are executed by remote processingdevices that are linked through a communications network. In thedistributed computing environments, program modules may be located inlocal or remote computing system storage media including storagedevices.

Exemplary Embodiments

FIG. 1 is a flow chart of an embodiment of a body contour key pointdetection method of the present disclosure. As shown in FIG. 1, themethod according to the embodiments includes: operations S100, S110, andS120. Each operation in FIG. 2 is illuminated in detail as follows.

S100, an image feature of an image block including a body is obtained;for example, an image feature of an image block including a body in animage to be processed is obtained. According to one or more embodimentsof the present disclosure, the body refers to human body.

According to one or more embodiments of the present disclosure, theimage to be processed in the present disclosure may be an image such asa picture or photo in a static state, and may also be a video frame in avideo in a dynamic state. The body in the image to be processed may be afront body, a side body, or a back body. The body in the image to beprocessed may be in multiple forms, for example, the body in the imageto be processed is in a walking, jumping, squatting, accumbent, orheadstand form. In addition, the body included in the image block of thepresent disclosure may be an intact body (for example, the body shown inFIG. 2), and may also be a partial body (i.e., a part of the body, e.g.,a half-length portrait). An expressive form of the body in the image tobe processed is not limited in the present disclosure.

According to one or more embodiments of the present disclosure, in thepresent disclosure, the image feature of the image block including thebody in the image to be processed may be obtained by means of a neuralnetwork. The neural network in the present disclosure for extracting theimage feature of the image block including the body in the image to beprocessed may be a Convolutional Neural Network (CNN). To bedistinguished from a first neural network of the present disclosure, theneural network for extracting the image feature of the image blockincluding the body in the image to be processed is referred to as asecond neural network in the present disclosure.

According to one or more embodiments of the present disclosure, in thepresent disclosure, the image block including the body in the image tobe processed may be input into the second neural network and the imagefeature of the image block is output by means of the second neuralnetwork. The size of the image block input into the second neuralnetwork is generally associated with a size requirement of an inputimage of the second neural network, for example, the size of the imageblock may be 256×256. The present disclosure does not limit the size ofthe image block.

According to one or more embodiments of the present disclosure, anetwork structure of the second neural network in the present disclosuremay be flexibly designed according to actual requirements for extractingthe image feature; the embodiment of the present disclosure does notlimit the network structure of the second neural network; for example,the second neural network may include, but not limited to, aconvolutional layer, a nonlinear Relu layer, a pooling layer, and a fullconnection layer. The more the number of layers included by the secondneural network, the deeper the network. Furthermore, for instance, thenetwork structure of the second neural network of the present disclosuremay adopt, but not limited to, network structures adopted by neuralnetworks such as ALexNet, a Deep Residual Network (ResNet), or VisualGeometry Group Network (VGGnet).

According to one or more embodiments of the present disclosure, theimage block in the present disclosure may be the full image of the imageto be processed, and may also be a partial image including the body inthe image to be processed. In addition, the image block may further bean image block obtained by processing the partial image including thebody in the image to be processed. The image block input into the secondneural network in the present disclosure may be an image block obtainedby cutting the image to be processed. According to one or moreembodiments of the present disclosure, the image to be processed may besubjected to body detection first and is cut according to a bodydetection result, so as to obtain the image block in the presentdisclosure according to a cutting result.

According to one or more embodiments of the present disclosure, toobtain the image block with a predetermined size, in the presentdisclosure, the image to be processed may first be scaled, and then thescaled image to be processed is cut to obtain the image block includinga body according to the predetermined size. In the present disclosure,the image to be processed can be scaled according to the body detectionresult, so that the cut image block including the body has thepredetermined size. The image to be processed can be scaled and cut bymeans of the neural network in the present disclosure. To bedistinguished from the first neural network and the second neuralnetwork in the present disclosure, the neural network for executingscaling and cutting in the present disclosure can be referred to as aninput neural network.

According to one or more embodiments of the present disclosure, in thepresent disclosure, the body detection result and the image to beprocessed can be provided to the input neural network, thereby obtainingthe image block including the body with the predetermined size accordingto the output of the input neural network in the present disclosure. Thebody detection result in the present disclosure generally refers toinformation that can represent the position of a body in the image to beprocessed. The body detection result of the image to be processed can beobtained by multiple means in the present disclosure, for example, bodydetection can be performed on the image to be processed using a bodydetector, thereby obtaining the body detection result. The presentdisclosure does not limit the implementing mode for body detection onthe image to be processed, the structure of the body detector, etc.

According to one or more embodiments of the present disclosure, the bodydetection result of the present disclosure may include: a centralposition of a body bounding box in the image to be processed and a bodyscale factor. The central position of the body bounding box in thepresent disclosure may be referred to as a body position or a bodycentral point. The body scale factor in the present disclosure may beused for determining the size of the body bounding box. The body scalefactor may be a scaling factor, for example, a body scale factor s maybe a scaling factor that enables a head size H in the image to beprocessed to be scaled to a standard size h, i.e., s=h/H. In the presentdisclosure the central position of the body bounding box in the image tobe processed, the body scale factor, and the image to be processed canbe provided to the input neural network; the image to be processed isscaled by means of the input neural network according to the body scalefactor and the scaled image to be processed is cut according to thepredetermined size and the central position, so as to output the imageblock including the body with the predetermined size.

According to one or more embodiments of the present disclosure,operation S100 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby an image feature obtaining module 800 run by the processor.

S110, a body contour key point prediction result of the body is obtainedby means of the first neural network according to the image feature.

According to one or more embodiments of the present disclosure, thefirst neural network in the present disclosure is obtained by using animage sample carried with body contour key point marking information fortraining. Under the condition of scaling and cutting the image to beprocessed by means of the input neural network and extracting the imagefeature of the image block including the body by means of the secondneural network in the present disclosure, the input neural network, thefirst neural network, and the third neural network in the presentdisclosure can all be obtained by using the image sample carried withthe body contour key point marking information for training. The processof training the neural network by using the image sample can be refer tothe following description for FIG. 3 and FIG. 4, and is not explained indetail herein.

According to one or more embodiments of the present disclosure, thefirst neural network in the present disclosure would respectively form abody contour key point response diagram for at least one body contourkey point regarding the input image feature. The body contour key pointresponse diagram in the present disclosure may represent: thepossibility for occurrences of a corresponding body contour key point inmultiple positions (e.g. each position) in the image block. The bodycontour key point response diagram is an optional representation formfor the body contour key point prediction result. According to one ormore embodiments of the present disclosure, the number of the bodycontour key points is set in advance as N (N is an integer greater than0), and thus the first neural network in the present disclosure wouldform N body contour key point response diagrams for the input imagefeature; each body contour key point response diagram corresponds to abody contour key point; different body contour key point responsediagrams correspond to different body contour key points; therefore, onebody contour key point response diagram may reflect the possibility foroccurrences of the body contour key point corresponding thereto in eachposition in the image block.

According to one or more embodiments of the present disclosure, the bodycontour key point response diagram output by the first neural networkmay be a body contour key point confidence diagram. The body contour keypoint confidence diagram may reflect confidence for occurrences of acorresponding body contour key point in multiple positions (e.g., eachposition) in the image block.

According to one or more embodiments of the present disclosure, theregenerally are multiple body contour key points in the presentdisclosure. According to one or more embodiments of the presentdisclosure, the number of the body contour key points is generally notless than 42, and generally no more than 115; for example, the number ofthe body contour key points in the present disclosure may be 64. Anoptional example of the distribution of the 64 body contour key pointsis as shown in FIG. 2.

According to one or more embodiments of the present disclosure, the bodycontour key point includes at least one of: a head contour key point, anarm contour key point, a hand contour key point, a shoulder contour keypoint, a leg contour key point, a waist contour key point, and a footcontour key point.

According to one or more embodiments of the present disclosure, thenumber of the head contour key points in the present disclosure isgenerally not less than 3, and generally no more than 8.

According to one or more embodiments of the present disclosure, the armcontour key point in the present disclosure may include: left side armcontour key points (including a left side arm inner contour key pointand/or a left side arm outer contour key point) and right side armcontour key points (including a right side arm inner contour key pointand/or a right side arm outer contour key point). The number of theleft/right side arm contour key points is generally not less than 6, andgenerally no more than 18.

According to one or more embodiments of the present disclosure, the handcontour key point in the present disclosure may include: left side handcontour key points and right side hand contour key points. The number ofthe left/right side hand contour key points is generally not less than2, and generally no more than 2.

According to one or more embodiments of the present disclosure, theshoulder contour key point in the present disclosure may include: leftside shoulder contour key points and right side shoulder contour keypoints. The number of the left/right side shoulder contour key points isgenerally not less than 2, and generally no more than 4.

According to one or more embodiments of the present disclosure, the legcontour key point in the present disclosure may include: left side legcontour key points (including a left side leg inner contour key pointand/or a left side leg outer contour key point) and right side legcontour key points (including a right side leg inner contour key pointand/or a right side leg outer contour key point). The number of theleft/right side leg contour key points is generally not less than 6, andgenerally no more than 18. The key point located at the crotch in theleft side leg contour key points and the right side leg contour keypoints is repeated.

According to one or more embodiments of the present disclosure, thewaist contour key point in the present disclosure may include: left sidewaist contour key points and right side waist contour key points. Thenumber of the left/right side waist contour key points is generally notless than 2, and generally no more than 10.

According to one or more embodiments of the present disclosure, the footcontour key point in the present disclosure may include: left side footcontour key points and right side foot contour key points. The number ofthe left/right side foot contour key points is generally not less than2, and generally no more than 2.

According to one or more embodiments of the present disclosure, the bodycontour key points in the present disclosure include:

3 head contour key points, respectively being: 1 head top key point, 1nose tip key point, and a chin key point;

10 left side arm contour key points, respectively being: 2 left wristcontour key points (1 inner and 1 outer key points, i.e., a left wristinner contour key point and a left wrist outer contour key point), 2left elbow contour key point (1 inner and 1 outer key points, i.e., aleft elbow inner contour key point and a left elbow outer contour keypoint), 2 left arm root contour key points (1 inner and 1 outer keypoints, i.e., a left arm root inner contour key point and a left armroot outer contour key point, where the left arm root inner contour keypoint is a left armpit key point), 2 lower left arm contour midpoint keypoints (1 inner and 1 outer key points, i.e., a lower left arm innercontour midpoint key point and a lower left outer contour midpoint keypoint) located at a midpoint position between the left wrist contour keypoints and the left elbow contour key points, and 2 upper left armcontour midpoint key points (1 inner and 1 outer key points, i.e., anupper left arm inner contour midpoint key point and an upper left armouter contour midpoint key point) located at a midpoint position betweenthe left elbow contour key points and the left arm root contour keypoints. The upper arm in the present disclosure is a brachium;

10 right side arm contour key points, respectively being: 2 right wristcontour key points (1 inner and 1 outer key points, i.e., a right wristinner contour key point and a right wrist outer contour key point), 2right elbow contour key point (1 inner and 1 outer key points, i.e., aright elbow inner contour key point and a right elbow outer contour keypoint), 2 right arm root contour key points (1 inner and 1 outer keypoints, i.e., a right arm root inner contour key point and a right armroot outer contour key point, where the right arm root inner contour keypoint is a right armpit key point), 2 lower right arm contour midpointkey points (1 inner and 1 outer key points, i.e., a lower right arminner contour midpoint key point and a lower right outer contourmidpoint key point) located at a midpoint position between the rightwrist contour key points and the right elbow contour key points, and 2upper right arm contour midpoint key points (1 inner and 1 outer keypoints, i.e., an upper right arm inner contour midpoint key point and anupper right arm outer contour midpoint key point) located at a midpointposition between the right elbow contour key points and the right armroot contour key points. The lower arm in the present disclosure is aforearm;

2 left side hand contour key points, respectively being: 1 left hand tipkey point and 1 left palm midpoint key point;

2 right side hand contour key points, respectively being: 1 right handtip key point and 1 right palm midpoint key point;

2 left side shoulder contour key points, respectively being: 1 leftshoulder and head intersection key point located at located at anintersection position of the left side shoulder and the head, and 1 leftshoulder contour midpoint key point located at a midpoint positionbetween the left arm root contour key points and the left shoulder andhead intersection key point;

2 right side shoulder contour key points, respectively being: 1 rightshoulder and head intersection key point located at located at anintersection position of the right side shoulder and the head, and 1right shoulder contour midpoint key point located at a midpoint positionbetween the right arm root contour key points and the right shoulder andhead intersection key point;

10 left side leg contour key points, respectively being: 1 crotch keypoint, 2 left knee contour key points (1 inner and 1 outer key points,i.e., a left knee inner contour key point and a left knee outer contourkey point), 2 left ankle contour key points (1 inner and 1 outer keypoints, i.e., a left ankle inner contour key point and a left ankleouter contour key point), 1 left thigh root outside contour key point, 2left shank contour midpoint key points (1 inner and 1 outer key points,i.e., a left shank inner contour midpoint key point and a left shankouter contour midpoint key point) located at a midpoint position betweenthe left knee contour key points and the left ankle contour key points,1 left thigh inner contour midpoint key point located at a midpointposition between the left knee inner contour key point and the crotchkey point, and 1 left thigh outer contour midpoint key point located ata midpoint position between the left knee outer contour key point andthe left thigh root outside contour key point;

10 right side leg contour key points, respectively being: 1 crotch keypoint (repeated with the crotch key point in the left side leg contourkey points), 2 right knee contour key points (1 inner and 1 outer keypoints, i.e., a right knee inner contour key point and a right kneeouter contour key point), 2 right ankle contour key points (1 inner and1 outer key points, i.e., a right ankle inner contour key point and aright ankle outer contour key point), 1 right thigh root outside contourkey point, 2 right shank contour midpoint key points (1 inner and 1outer key points, i.e., a right shank inner contour midpoint key pointand a right shank outer contour midpoint key point) located at amidpoint position between the right knee contour key points and theright ankle contour key points, 1 right thigh inner contour midpoint keypoint located at a midpoint position between the right knee innercontour key point and the crotch key point, and 1 right thigh outercontour midpoint key point located at a midpoint position between theright knee outer contour key point and the right thigh root outsidecontour key point;

5 left side waist contour key points, respectively being: 5 equaldivision points generated by dividing the left thing root outsidecontour key point and the left arm root contour key point into 6 equalparts;

5 right side waist contour key points, respectively being: 5 equaldivision points generated by dividing the right thing root outsidecontour key point and the right arm root contour key point into 6 equalparts;

2 left side foot contour key points, respectively being: 1 left tiptoekey point and 1 left heel key point;

2 right side foot contour key points, respectively being: 1 right tiptoekey point and 1 right heel key point;

The total number of the body contour key points above is 64 (removing arepeated crotch key point). As can be known by means of multipleexperiments, 64 body contour key points are detected in the presentdisclosure without consuming excessive computing resources and timecosts. As can be seen from FIG. 2 that, 64 body contour key points mayaccurately draw an outer contour of a body. In view of the above, the 64body contour key points may obtain a better balance among aspects ofresource consuming, instantaneity, and accuracy of drawing an outercontour of a body.

The detailed distribution of the body contour key points is only anoptional example. The present disclosure does not limit the distributioncondition of the body contour key points as long as the outer contour ofthe body can be basically drawn by means of the body contour key points.In addition, the greater the number of the body contour key points (forexample, more than 64), the more accurate the body contour drawn by thebody contour key points.

According to one or more embodiments of the present disclosure,operation S110 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby a prediction result obtaining module 810 run by the processor.

S120, a body contour key point in the image block is obtained accordingto the body contour key point prediction result.

According to one or more embodiments of the present disclosure,according to a predetermined body contour point determining condition,information output for the first neural network can be determined, andthe body contour key point in the image block can be obtained accordingto the determining result, so that the body contour key point in theimage block is transferred into the image to be processed, therebyobtaining the body contour key point in the image to be processed. Thedetermining condition in the present disclosure is set according toactual conditions, for example, the determining condition is set basedon the body contour key point response diagram.

According to one or more embodiments of the present disclosure, for Nbody contour key points, the first neural network outputs N body contourkey point confidence diagrams; in the present disclosure, determiningmay be executed for the M-th (M is greater than o and less than or equalto N) body contour key point confidence diagram according to apredetermined confidence requirement, so as to select a point meetingthe predetermined confidence requirement from the body contour key pointconfidence diagrams, for example, in the present disclosure, thefollowing operations may be executed for the M-th body contour key pointconfidence diagram according to the predetermined confidencerequirement: selecting a point with the highest confidence from the M-thbody contour key point confidence diagram; determining whether the pointwith the highest confidence is greater than a predetermined confidencethreshold; if not greater than the predetermined confidence threshold,indicating that the M-th body contour key point does not exist in theimage to be processed; moreover, if greater than the predeterminedconfidence threshold, performing mapping processing on the position ofthe point in the image block, so as to obtain a mapping position pointof the point in the image to be processed; the mapping position pointbeing the M-th body contour key point.

According to one or more embodiments of the present disclosure,operation S120 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby a contour key point obtaining module 820 run by the processor.

The body contour key point obtained in the present disclosurefacilitates implementing of accurate drawing of the body outer contourin the image to be processed, thereby facilitating obtaining richer bodysemantic information from the image including the body, so as to providericher body semantic information for a corresponding application, forexample, the body contour key point extracted from the presentdisclosure can be used for applications such as sticker processing, bodyaction analysis, and body morphology analysis In the present disclosure,the body contour key point of the image to be processed is obtained bymeans of the first neural network, facilitating rapid and accurateobtaining of the body contour key point of the image to be processed, soas to facilitate meeting the requirement of extracting the body contourkey point of the image to be processed in real time, so that thetechnical solution provided by the present disclosure is advantageouslyapplied to real-time environments such as live-broadcasting orman-machine interaction.

FIG. 3 is a flow chart of an embodiment of the neural network trainingmethod of the present disclosure. As shown in FIG. 3, the methodaccording to the embodiments includes: operations S300, S310, and S320.Each operation in FIG. 3 is illuminated in detail as follows.

S300, the image feature of the sample image block including the body isobtained, for example, obtaining an image sample from a training dataset; and obtaining an image feature of the sample image block includingthe body in the image sample. According to one or more embodiments ofthe present disclosure, the training data set in the present disclosureincludes multiple image samples for training the first neural network;under a normal condition, each image sample is provided with bodycontour key point marking information; for example, each image sample isprovided with multiple pieces of body contour key point (e.g., 64 orgreater than 64 and less than or equal to 115 body contour key pointsetc.) marking information. In the present disclosure, one or more imagesamples may be read from the training data set at one time according toa random reading mode or an image sample arrangement order sequencereading mode. The method for generating the body contour key pointmarking information in the image sample in the training data set can bedescribed as shown FIG. 5, and is omitted herein.

According to one or more embodiments of the present disclosure, multiplemodes can be adopted to obtain the image feature of the sample imageblock including the body, for example, in the present disclosure, theimage feature of the image block including the body in the image samplecan be obtained by using the neural network. According to one or moreembodiments of the present disclosure, body detection may be performedon the image sample first; then the body detection result and the imagesample are provided to the input neural network to be trained to obtaina sample image block including the body with a predetermined size bymeans of the input neural network to be trained; and finally, the imagefeature of the sample image block is obtained by means of the secondneural network to be trained. The body detection result may be thecentral position of the body bounding box and the body scale factor inthe image sample.

According to one or more embodiments of the present disclosure,operation S300 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby a first module 900 run by the processor.

S310, a body contour key point prediction result of the body is obtainedby means of a first neural network to be trained according to the imagefeature.

According to one or more embodiments of the present disclosure, thefirst neural network to be trained in the present disclosure wouldrespectively form a body contour key point response diagram for at leastone body contour key point regarding the input image feature of thesample image block. According to one or more embodiments of the presentdisclosure, the number of the body contour key points is set in advanceas N (N is an integer greater than 0), and thus the first neural networkto be trained in the present disclosure would form N body contour keypoint response diagrams for the input image feature of the sample imageblock; each body contour key point response diagram corresponds to abody contour key point; different body contour key point responsediagrams correspond to different body contour key points; therefore, onebody contour key point response diagram may reflect the possibility foroccurrences of the body contour key point corresponding thereto in eachposition in the sample image block.

According to one or more embodiments of the present disclosure, the bodycontour key point response diagram output by the first neural network tobe trained may be a body contour key point confidence diagram. The bodycontour key point confidence diagram may reflect confidence foroccurrences of a corresponding body contour key point in multiplepositions (e.g., each position) in the sample image block.

According to one or more embodiments of the present disclosure,operation S310 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby a second module 910 run by the processor.

S320, a difference between the body contour key point prediction resultand the body contour key point marking information are used as guidanceinformation to perform supervised learning on the first neural networkto be trained.

According to one or more embodiments of the present disclosure, for anypiece of the body contour key point marking information of the imagesample, in the present disclosure, the body contour key point responsediagram (e.g., the body contour key point confidence diagram) can begenerated for the body contour key point marking information. For partor all of the body contour key point marking information of the imagesample, in the present disclosure, one body contour key point responsediagram (e.g., the body contour key point confidence diagram) can begenerated respectively. Furthermore, in the present disclosure, thedifference between the body contour key point response diagram (e.g.,the body contour key point confidence diagram) output by the firstneural network to be trained and the generated body contour key pointresponse diagram (e.g., the body contour key point confidence diagram)as the guidance information to perform supervised learning on the firstneural network to be trained.

According to one or more embodiments of the present disclosure, the N(e.g., 64) body contour key point confidence diagrams output by thefirst neural network to be trained for N (e.g., 64) body contour keypoints are respectively: a first output body contour key pointconfidence diagram, a second output body contour key point confidencediagram, . . . , and an N-th output body contour key point confidencediagram; the N (e.g., 64) body contour key point confidence diagramsgenerated by the present disclosure for N (e.g., 64) pieces of bodycontour key point marking information of the image same arerespectively: a first generated body contour key point confidencediagram, a second generated body contour key point confidence diagram, .. . , and an N-th generated body contour key point confidence diagram;the difference between the first output body contour key pointconfidence diagram and the first generated body contour key pointconfidence diagram, the difference between the second output bodycontour key point confidence diagram and the second generated bodycontour key point confidence diagram, . . . , and the N-th output bodycontour key point confidence diagram and the N-th generated body contourkey point confidence diagram, as the guidance information to performsupervised learning on the first neural network to be trained.

According to one or more embodiments of the present disclosure, the bodycontour key point confidence diagram can be generated by multiple means,for example, for any piece of body contour key point marking informationof the image sample, the position thereof in the sample image block isfirst determined according to the body contour key point markinginformation; then, a Gaussian response is formed in a predeterminedperipheral region of the position (e.g., an X×Y using the position asthe center, and e.g., the entire image block region), so that the bodycontour key point confidence diagram corresponding to the body contourkey point marking information can be obtained according to the Gaussianresponse. Using the method above, in the present disclosure, for eachbody contour key point marking information of the image sample, a bodycontour key point confidence diagram with a size equal to that of thesample image block can be generated respectively. In addition, in thepresent disclosure, other modes can also be adopted to generate the bodycontour key point confidence diagram, for example, setting the positionin the sample image corresponding to the body contour key point markinginformation as 1, while setting other positions as 0, thereby obtaininga body contour key point confidence diagram. The present disclosure doesnot limit the implementing mode of generating the body contour key pointconfidence diagram.

According to one or more embodiments of the present disclosure,according to a predetermined body contour point determining condition,information output for the first neural network to be trained can bedetermined, and the body contour key point in the sample image block canbe obtained according to the determining result, so that the bodycontour key point in the sample image block is transferred into theimage sample in the present disclosure, thereby obtaining the bodycontour key point in the image sample. Then in the present disclosure,for the purpose of reducing the difference between the obtained bodycontour key point in the image sample and the body contour key pointmarking information of the image sample, the network parameter (e.g.,weight) in the first neural network to be trained can be adjusted toperform supervised learning on the first neural network to be trained.

According to one or more embodiments of the present disclosure, when thetraining for the first neural network to be trained meets apredetermined iteration condition, this training process ends. Thepredetermined iteration condition in the present disclosure may include:the difference between the information output by the first neuralnetwork to be trained and the body contour key point marking informationof the image sample meets a predetermined difference requirement. Whenthe difference meets the predetermined difference requirement, thistraining for the first neural network to be trained successfully ends.The predetermined iteration condition in the present disclosure may alsoinclude: training the first neural network to be trained, the number ofused image samples meeting a predetermined number requirement, etc.Under the condition that the number of used image samples meets thepredetermined number requirement but the difference does not meet thepredetermined difference condition, this training for the first neuralnetwork to be trained does not succeed. The first neural network withthe successfully completed training may be used for performing bodycontour key point detection processing on the image feature of the imageblock including the body in the image to be processed.

According to one or more embodiments of the present disclosure,operation S320 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby a third module 920 run by the processor.

FIG. 4 is a flow chart of another embodiment of the neural networktraining method of the present disclosure. As shown in FIG. 4, themethod according to the embodiments includes: operations S400, S410,S420, S430, S440, and S450. Each operation in FIG. 4 is illuminated asfollows.

S400, an image sample is obtained from a training data set. The imagesample in the present disclosure may be described as in operation S300above.

S410, the image sample is provided to a body detector to obtain the bodydetection result of the image sample. According to one or moreembodiments of the present disclosure, the body detector can be used toobtain the central position of the body bounding box and the body scalefactor in the image sample.

S420, a body detection result and the image sample are provided to aninput neural network to be trained, to obtain a sample image blockincluding the body with a predetermined size by means of the inputneural network to be trained.

According to one or more embodiments of the present disclosure, theinput neural network to be trained may determine, according to thecentral position of the body bounding box and the body scale factor inthe input image sample, the size and position of the body bounding boxin the image sample, so that the input neural network to be trained maycut the input image sample to obtain the sample image block according tothe size and position of the body bounding box and output the sampleimage block.

S430, the sample image block is provided to the second neural network tobe trained, to obtain the image feature of the sample image block bymeans of the second neural network. The structure of the second neuralnetwork to be trained and the like may refer to related description inthe method embodiments above, and are not repeatedly explained herein.

According to one or more embodiments of the present disclosure,operations S400, S410, S420, and S430 may be executed by invoking, by aprocessor, corresponding instructions stored in a memory, and may alsobe executed by the first module 900 run by the processor.

S440, the image feature of the sample image block is provided to thefirst neural network to be trained, to perform body contour key pointprediction processing according to the image feature of the sample imageblock by means of the first neural network to be trained, so as toobtain at least one body contour key point response diagram.

According to one or more embodiments of the present disclosure, for theinput image feature of the sample image block, the first neural networkto be trained may respectively output a body contour key pointconfidence diagram for each body contour key point, i.e., each outputbody contour key point confidence diagram corresponds to one differentbody contour key point; one body contour key point confidence diagrammay represent: the confidence for the occurrence of each position of thebody contour key point corresponding thereto in the sample image block.

According to one or more embodiments of the present disclosure,operation S440 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby the second module 910 run by the processor.

S450, the difference between the N body contour key point responsediagrams output by the first neural network to be trained and the N bodycontour key point response diagrams formed based on the body contour keypoint marking information of the image sample is used as the guidanceinformation to perform supervised learning on the input neural networkto be trained, the first neural network to be trained, and the secondneural network to be trained.

According to one or more embodiments of the present disclosure, for anypiece of body contour key point marking information of the image sample,the position thereof in the sample image block can be first determinedaccording to the body contour key point marking information; then, aGaussian response is formed in a predetermined peripheral region of theposition (e.g., an X×Y using the position as the center, and e.g., theentire image block region), so that the body contour key pointconfidence diagram corresponding to the body contour key point markinginformation can be obtained according to the Gaussian response. Usingthe method above, in the present disclosure, for each body contour keypoint marking information of the image sample, a body contour key pointconfidence diagram with a size equal to that of the sample image blockcan be generated respectively. In this way, in the present disclosure,according to the difference between the body contour key pointconfidence diagram output by the first neural network for each bodycontour key point and the generated corresponding body contour key pointconfidence diagram being used as the guiding information, supervisedlearning is performed on the input neural network to be trained, thefirst neural network to be trained, and the second neural network to betrained That is to say, in the present disclosure, for the purpose ofreducing the difference between the generated body contour key pointconfidence diagram and the body contour key point confidence diagramoutput by the first neural network to be trained, supervised learningcan be performed on the network parameters (e.g., weight) in the inputneural network to be trained, the first neural network to be trained,and the second neural network to be trained.

When the training for the input neural network to be trained, the firstneural network to be trained, and the second neural network to betrained meets the predetermined iteration condition (e.g., thedifference meets the predetermined difference requirement or the numberof the used image samples meets the predetermined number requirement),this training process ends. When the training process ends, if thedifference between the information output by the first neural network tobe trained and the body contour key point marking information of theimage sample meets the predetermined difference requirement, thistraining for the input neural network to be trained, the first neuralnetwork to be trained, and the second neural network to be trainedcompletes successfully; or this training for the input neural network tobe trained, the first neural network to be trained, and the secondneural network to be trained does not succeed.

According to one or more embodiments of the present disclosure,operation S450 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby the third module 920 run by the processor.

FIG. 5 is a flow chart of an embodiment of a body contour key pointmarking information generating method of an image sample of the presentdisclosure. As shown in FIG. 5, the method according to the embodimentsincludes: operations S500, S510, and S520. According to one or moreembodiments of the present disclosure, the method according to theembodiments further includes: operation S530. Each operation in FIG. 5is illuminated as follows.

S500, a body skeleton key point of the image sample is obtained.

According to one or more embodiments of the present disclosure, multiplemodes can be used to obtain the body skeleton key point of the imagesample; According to one or more embodiments of the present disclosure,the image sample is first provided to the body detector to obtain a bodydetection result of the image sample, for example, obtaining the centralposition of the body bounding box and the body scale factor in the imagesample; then the body detection result of the image sample and the imagesample are provided to the input neural network, to obtain the sampleimage block including the body in the image sample by means of the inputneural network; then, the sample image block is provided to the bodyskeleton detection module to obtain the body skeleton key point in thesample image block by means of the body skeleton detection module.

According to one or more embodiments of the present disclosure, the bodyskeleton key point in the present disclosure is generally located at acentral position of a body joint. Using the body skeleton key pointcannot draw the body outer contour. According to one or more embodimentsof the present disclosure, the body skeleton key point obtained by thepresent disclosure includes: a right shoulder key point, a right elbowkey point, a right wrist key point, a left shoulder key point, a leftelbow key point, a left wrist key point, a right hip key point, a rightknee key point, a right ankle key point, a left hip key point, a leftknee key point, a left ankle key point, a head top key point, a neck keypoint, etc. The present disclosure does not limit the number of the bodyskeleton key points and the implementing mode for obtaining the bodyskeleton key points.

S510, an auxiliary line is configured according to the body contour keypoint marking information in the first set and/or the body skeleton keypoint of the image sample based on a predetermined auxiliary lineconfiguring mode.

According to one or more embodiments of the present disclosure, the bodycontour key point marking information may include the serial number andposition coordinate of the body contour key point. An image sample mayinclude multiple sets; different sets may correspond to differentmarking difficulties; the body contour key point marking information ina set with a high marking difficulty may be formed based on the bodycontour key point marking information in a set with a low markingdifficulty. High and low marking difficulties in the present disclosurecan be distinguished based on whether the position of the body contourkey point is easy to be accurately determined. The marking difficultyfor the body contour key point marking information in the first set ofthe image sample in the present disclosure is generally the lowest,i.e., the body contour key point marking information that would beaccurately determined in an easiest manner is configured in the firstset. The body contour key point marking information in the first set canbe formed by means of manual marking in the present disclosure.

According to one or more embodiments of the present disclosure, the bodycontour key point marking information in the first set may include:

head contour key point marking information, such as 1 piece of head topkey point marking information, 1 piece of nose tip key point markinginformation, and 1 piece of chin key point marking information;

left side hand contour key point marking information, such as 1 piece ofleft hand tip key point marking information and 1 piece of left palmmidpoint key point marking information;

right side hand contour key point marking information, such as 1 pieceof right hand tip key point marking information and 1 piece of rightpalm midpoint key point marking information;

left side wrist contour key point marking information (1 inner and 1outer pieces of information), left elbow contour key point markinginformation (1 inner and 1 outer pieces of information), and 1 piece ofleft arm root inner contour key point marking information (i.e., leftarmpit contour key point marking information) in left side arm contourkey point marking information;

right side wrist contour key point marking information (1 inner and 1outer pieces of information), right elbow contour key point markinginformation (1 inner and 1 outer pieces of information), and 1 piece ofright arm root inner contour key point marking information (i.e., rightarmpit contour key point marking information) in right side arm contourkey point marking information;

1 piece of left shoulder and head intersection key point markinginformation in left side shoulder contour key point marking information;

1 piece of right shoulder and head intersection key point markinginformation in right side shoulder contour key point markinginformation;

1 pieces of crotch key point marking information, left knee contour keypoint marking information (1 inner and 1 outer pieces of information),and left ankle contour key point marking information (1 inner and 1outer pieces of information) in left side leg contour key point markinginformation;

1 pieces of crotch key point marking information (repeated), right kneecontour key point marking information (1 inner and 1 outer pieces ofinformation), and right ankle contour key point marking information (1inner and 1 outer pieces of information) in right side leg contour keypoint marking information;

left side foot contour key point marking information, such as 1 piece ofleft tiptoe key point marking information and 1 piece of left heel keypoint marking information;

right side foot contour key point marking information, such as 1 pieceof right tiptoe key point marking information and 1 piece of right heelkey point marking information.

According to one or more embodiments of the present disclosure, theconfiguring mode for the predetermined auxiliary line generallyincludes: disposing a vertical line vertical to a connection linebetween two body skeleton key points by means of a body contour keypoint.

In a first optional example, a connection line (referred to as a firstconnection line hereinafter) between the left shoulder key point and theleft elbow key point in the body skeleton key points is made; by meansof the left armpit key point in the first set, a vertical line (referredto as a first vertical line hereinafter) of the first connection line isconfigured; the first vertical line is an auxiliary line in the presentdisclosure. Similarly, a connection line (referred to as a secondconnection line hereinafter) between the right shoulder key point andthe right elbow key point in the body skeleton key points is made; bymeans of the right armpit key point in the first set, a vertical line(referred to as a second vertical line hereinafter) of the secondconnection line is configured; the second vertical line is an auxiliaryline in the present disclosure.

In a second optional example, a connection line (referred to as a thirdconnection line hereinafter) between the left hip key point and the leftknee key point in the body skeleton key points is made; by means of thecrotch key point in the first set, a vertical line (referred to as athird vertical line hereinafter) of the third connection line isconfigured; the third vertical line is an auxiliary line in the presentdisclosure. Similarly, a connection line (referred to as a fourthconnection line hereinafter) between the right hip key point and theright knee key point in the body skeleton key points is made; by meansof the crotch key point in the first set, a vertical line (referred toas a fourth vertical line hereinafter) of the fourth connection line isconfigured; the fourth vertical line is an auxiliary line in the presentdisclosure.

According to one or more embodiments of the present disclosure,operation S510 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby a body contour key point detection apparatus 1000 run by theprocessor.

S520, body contour key point marking information in a second set isformed according to a point selected from the auxiliary line.

According to one or more embodiments of the present disclosure, themarking difficulty for the body contour key point marking information inthe second set of the image sample of the present disclosure isgenerally higher than that in the first set. The body contour key pointmarking information in the second set is generally formed based on thebody contour key point marking information in the first set. The bodycontour key point marking information in the second set can be formed bymeans of selecting a point from the auxiliary line, for example,selecting an intersection point, in the present disclosure. Forming thebody contour key point marking information in the second set by means ofmaking an auxiliary line and selecting a point from the auxiliary linefacilitates improving the accuracy and consistence of the body contourkey point marking information in the second set.

According to one or more embodiments of the present disclosure, anintersection point between the first vertical line and the body outercontour in the sample image block can be used as the left arm root outercontour key point, and the position coordinate of the left arm rootouter contour key point in the sample image block is converted to theposition coordinate in the image sample, so that the left arm root outercontour key point marking information in the second set can be formedaccording to the serial number of the left arm root outer contour keypoint and the position coordinate thereof in the image sample. This modefacilitates improving the accuracy and consistence of the left arm rootouter contour key point marking information

According to one or more embodiments of the present disclosure, anintersection point between the second vertical line and the body outercontour in the sample image block can be used as the right arm rootouter contour key point, and the position coordinate of the right armroot outer contour key point in the sample image block is converted tothe position coordinate in the image sample, so that the right arm rootouter contour key point marking information in the second set can beformed according to the serial number of the right arm root outercontour key point and the position coordinate thereof in the imagesample. This mode facilitates improving the accuracy and consistence ofthe right arm root outer contour key point marking information

According to one or more embodiments of the present disclosure, anintersection point between the third vertical line and the body outercontour in the sample image block can be used as the left thigh rootouter contour key point, and the position coordinate of the left thighroot outer contour key point in the sample image block is converted tothe position coordinate in the image sample, so that the left thigh rootouter contour key point marking information in the second set can beformed according to the serial number of the left thigh root outercontour key point and the position coordinate thereof in the imagesample. This mode facilitates improving the accuracy and consistence ofthe left thigh root outer contour key point marking information

According to one or more embodiments of the present disclosure, anintersection point between the fourth vertical line and the body outercontour in the sample image block can be used as the right thigh rootouter contour key point, and the position coordinate of the right thighroot outer contour key point in the sample image block is converted tothe position coordinate in the image sample, so that the right thighroot outer contour key point marking information in the second set canbe formed according to the serial number of the right thigh root outercontour key point and the position coordinate thereof in the imagesample. This mode facilitates improving the accuracy and consistence ofthe right thigh root outer contour key point marking information

According to one or more embodiments of the present disclosure,operations S300, S510, and S520 may be executed by invoking, by aprocessor, corresponding instructions stored in a memory, and may alsobe executed by a first marking module 930 run by the processor.

S530, body contour key point marking information in a third set isformed according to N1 division points on a connection line between twobody contour key points in the first set and/or the second set.

According to one or more embodiments of the present disclosure, themarking difficulty for the body contour key point marking information inthe third set of the image sample of the present disclosure is generallyhigher than that in the first set. However, the marking difficulty inthe third set would not necessarily be higher than that in the secondset. According to one or more embodiments of the present disclosure, thebody contour key point marking information in the present disclosure maybe marking information of N1 division points on the connection linebetween two body contour key points in the first set, may also bemarking information of N1 division points on the connection line betweentwo body contour key points in the second set, and may further bemarking information of N1 division points on the connection line betweenone body contour key point in the first set and one body contour keypoint in the second set. The N1 division points in the presentdisclosure generally are N1 even division points; the N1 even divisionpoints are N1 equal division points; and N1 is an integer greater than1, for example, 2 equal division points (i.e., a middle point), 5 equaldivision points, or 6 equal division points.

According to one or more embodiments of the present disclosure, a middlepoint on the connection line between the left wrist inner contour keypoint and the left elbow inner contour key point can be used as thelower left arm inner contour midpoint key point and the lower left arminner contour midpoint key point marking information in the third set isformed. In the present disclosure, a midpoint point on the connectionline between the left wrist outer contour key point and the left elbowouter contour key point can be used as the lower left arm outer contourmidpoint key point and the lower left arm outer contour midpoint keypoint marking information in the third set is formed.

According to one or more embodiments of the present disclosure, a middlepoint on the connection line between the right wrist inner contour keypoint and the right elbow inner contour key point can be used as thelower right arm inner contour midpoint key point and the lower right arminner contour midpoint key point marking information in the third set isformed. In the present disclosure, a middle point on the connection linebetween the right wrist outer contour key point and the right elbowouter contour key point can be used as the lower right arm outer contourmidpoint key point and the lower right arm outer contour midpoint keypoint marking information in the third set is formed.

According to one or more embodiments of the present disclosure, a middlepoint on the connection line between the left elbow inner contour keypoint and the left arm root inner contour key point (i.e., the leftarmpit key point) can be used as the upper left arm inner contourmidpoint key point and the upper left arm inner contour midpoint keypoint marking information in the third set is formed. In the presentdisclosure, a middle point on the connection line between the left elbowouter contour key point and the left arm root outer contour key pointcan be used as the upper left arm outer contour midpoint key point andthe upper left arm outer contour midpoint key point marking informationin the third set is formed.

According to one or more embodiments of the present disclosure, a middlepoint on the connection line between the right elbow inner contour keypoint and the right arm root inner contour key point (i.e., the rightarmpit key point) can be used as the upper right arm inner contourmidpoint key point and the upper right arm inner contour midpoint keypoint marking information in the third set is formed. In the presentdisclosure, a middle point on the connection line between the rightelbow outer contour key point and the right arm root outer contour keypoint can be used as the upper right arm outer contour midpoint keypoint and the upper right arm outer contour midpoint key point markinginformation in the third set is formed.

According to one or more embodiments of the present disclosure, a middlepoint on the connection line between the left arm root outer contour keypoint and the left shoulder and head intersection key point can be usedas the left shoulder contour midpoint key point and the left shouldercontour midpoint key point marking information in the third set isformed.

According to one or more embodiments of the present disclosure, a middlepoint on the connection line between the right arm root outer contourkey point and the right shoulder and head intersection key point can beused as the right shoulder contour midpoint key point and the rightshoulder contour midpoint key point marking information in the third setis formed.

According to one or more embodiments of the present disclosure, a middlepoint on the connection line between the left knee inner contour keypoint and the left ankle inner contour key point can be used as the leftshank inner contour midpoint key point and the left shank inner contourmidpoint key point marking information in the third set is formed. Inthe present disclosure, a middle point on the connection line betweenthe left knee outer contour key point and the left ankle outer contourkey point can be used as the left shank outer contour midpoint key pointand the left shank outer contour midpoint key point marking informationin the third set is formed.

According to one or more embodiments of the present disclosure, a middlepoint on the connection line between the right knee inner contour keypoint and the right ankle inner contour key point can be used as theright shank inner contour midpoint key point and the right shank innercontour midpoint key point marking information in the third set isformed. In the present disclosure, a middle point on the connection linebetween the right knee outer contour key point and the right ankle outercontour key point can be used as the right shank outer contour midpointkey point and the right shank outer contour midpoint key point markinginformation in the third set is formed.

According to one or more embodiments of the present disclosure, a middlepoint on the connection line between the left knee inner contour keypoint and the left thigh root inner contour key point (i.e., the crotchkey point) can be used as the left thigh inner contour midpoint keypoint and the left thigh inner contour midpoint key point markinginformation in the third set is formed. In the present disclosure, amiddle point on the connection line between the left knee outer contourkey point and the left thigh root outer contour key point can be used asthe left shank outer contour midpoint key point and the left shank outercontour midpoint key point marking information in the third set isformed.

According to one or more embodiments of the present disclosure, i amiddle point on the connection line between the right knee inner contourkey point and the right thigh root inner contour key point (i.e., thecrotch key point) can be used as the right thigh inner contour midpointkey point and the right thigh inner contour midpoint key point markinginformation in the third set is formed. In the present disclosure, amiddle point on the connection line between the right knee outer contourkey point and the right thigh root outer contour key point can be usedas the right shank outer contour midpoint key point and the right shankouter contour midpoint key point marking information in the third set isformed.

According to one or more embodiments of the present disclosure, theconnection line between the left thigh root outside contour key pointand the left arm root contour key point (i.e., the left armpit keypoint) can be divided into 6 equal parts, thereby generating 5 equaldivision points; in the present disclosure, 5 equal division points canbe used as 5 left side waist contour key points and 5 pieces of the leftside waist contour key point marking information are formed in the thirdset.

According to one or more embodiments of the present disclosure, theconnection line between the right thigh root outside contour key pointand the right arm root contour key point (i.e., the right armpit keypoint) can be divided into 6 equal parts, thereby generating 5 equaldivision points; in the present disclosure, 5 equal division points canbe used as 5 right side waist contour key points and 5 pieces of theright side waist contour key point marking information are formed in thethird set.

According to one or more embodiments of the present disclosure,operation S530 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby a second marking module 940 run by the processor.

FIG. 6 is a flow chart of an embodiment of an image processing method ofthe present disclosure. As shown in FIG. 6, the method according to theembodiments includes: operations S600 and S610. Each operation in FIG. 6is illuminated as follows.

S600, the body contour key point of the image block including the bodyis detected. The body contour key point in the present disclosure isused for representing an outer contour of the body. In the presentdisclosure, detecting the body contour key point can be executed byusing each operation shown in FIG. 1. Descriptions are not made hereinrepeatedly.

According to one or more embodiments of the present disclosure,operation S600 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby a body contour key point detection apparatus 1000 run by theprocessor.

S610, an Augmented Reality (AR) effect rendering processing is performedon the image block according to the body contour key point.

According to one or more embodiments of the present disclosure, in thepresent disclosure, the position relation between the body contour keypoint of the image block and the sticker material can be determinedfirst, and then, according to the position relation and based on thesticker material, the AR effect rendering processing is performed on theimage block. According to one or more embodiments of the presentdisclosure, first, a first interface function used for creating asticker handle is invoked to create the sticker handle; next, a secondinterface function used for reading the sticker material is invoked tostore a sticker material compressed package into a memory by means ofreading and to analyze, thereby obtaining information required forrendering, such as, the body contour key point, the playback triggeringcondition of the sticker material, the playback state parameter of thesticker material according to analysis; and then a third interfacefunction used for rendering the sticker material is invoked to determinea position relation between the body contour key point of the imageblock and the sticker material according to an analysis result and toperform the AR effect rendering on the sticker material compressedpackage stored into the memory by means of reading on the image blockbased on the sticker handle according to the analysis result and theposition relation. After rendering is completed, the sticker handle isdestroyed.

According to one or more embodiments of the present disclosure, theplayback state parameter of the sticker material in the presentdisclosure may include: a layer where playback is located, a loopplayback parameter, and zooming in/out playback. The AR effect in thepresent disclosure may be a clothing effect (e.g., a clothing changingeffect), an accessory effect (e.g., an accessory wearing effect), anadvertisement effect, a 2D/3D special effect, etc. The presentdisclosure does not limit the content included in the playback stateparameter and the representation form of the AR effect.

According to one or more embodiments of the present disclosure,operation S610 may be executed by invoking, by a processor,corresponding instructions stored in a memory, and may also be executedby a rendering module 1010 run by the processor.

FIG. 7 is a flow chart of an embodiment of sticker processing performedon an image to be processed of the present disclosure. As shown in FIG.7, the method according to the embodiments includes: operations S700,S710, S720, S730, S740, and S750. Each operation in FIG. 7 isilluminated as follows.

S700, a sticker handle is created.

According to one or more embodiments of the present disclosure, thecreation of the sticker handle can be implemented by invoking aninterface function for creating the sticker handle. A correspondingstorage space is reserved for the created sticker handle, so as to storethe video frame and the sticker material. In addition, the stickerhandle can be created in a non-Graphics Library (non-GL) environment inthe present disclosure. The created sticker handle in the presentdisclosure may be the sticker handle that creates a resource in the GLbased on delay; that is to say, if the created sticker handle needs touse the resource in the GL, a strategy of creating when using theresource can be adopted, for example, before the first rendering,creating a corresponding resource in the GL environment.

S710, the sticker material is read.

According to one or more embodiments of the present disclosure, i thereading of the sticker material can be implemented by invoking theinterface function for reading the sticker material, for example,providing path information of the sticker material to the interfacefunction, reading the corresponding sticker material compressed packagein the memory, and analyzing the file in the compressed package in thememory. The sticker material compressed package generally includes ajson file; analysis of the json file may obtain information of thesticker material such as the number and size of pictures of the stickermaterial (for example, a 2D sticker and the like) in the stickermaterial compressed package, so as to facilitate reading of the stickermaterial on one hand, and may obtain position relations betweendifferent sticker materials and the body contour key point and atriggering action of the sticker materials on the other hand. Stickermaterial rendering information such as the sticker materials, theposition relations, and the triggering actions can be stored in thecreated sticker handle in the present disclosure.

According to one or more embodiments of the present disclosure, theframe number requiring to be rendered can be obtained according to thesticker rendering information in the sticker handle, and thecorresponding video frame is read in advance. In the present disclosure,a resource reading thread can be opened by means of a background tocomplete the operation of reading the video frame.

S720, the sticker material is rendered.

According to one or more embodiments of the present disclosure, therendering of the sticker material can be implemented by invoking aninterface function for rendering the sticker material. For example, theinterface function may determine, according to the sticker materialrendering information and the detected action in the video frame,whether requiring to render a corresponding sticker material in thevideo frame (for example, whether the action in the video frame belongsto a respective triggering action for each sticker material, etc.); andunder the condition that it is determined that the corresponding stickermaterial is required to be rendered in the video frame, the stickermaterial enters a playback state; in the present disclosure, thecorresponding sticker material can be displayed on a correspondingposition in a certain amount of video frames according to informationsuch as the position relation between the sticker material and the bodycontour key point. According to one or more embodiments of the presentdisclosure, a position matrix of the sticker materials in the playbackstate can be calculated, and stored in the sticker handle; duringdisplaying, the corresponding sticker material can be first transferredas an image texture in a video memory, to facilitate processing by adisplay processor; then according to the obtained position matrix, theposition of the image texture in the video frame is determined andrendered.

S730, the sticker handle is destroyed.

According to one or more embodiments of the present disclosure, thecreated sticker handle can be destroyed by invoking the interfacefunction for destroying the sticker handle under the condition that nocontinuous rendering of the sticker material is required, so as torelease a corresponding resource occupied by a feature handle.

According to one or more embodiments of the present disclosure,operations S700, S710, S720, and S730 may be executed by invoking, by aprocessor, corresponding instructions stored in a memory, and may alsobe executed by a rendering module 1010 run by the processor.

Any method provided by the embodiments of the present disclosure isexecuted by any appropriate device having data processing capability,including, but not limited to, a terminal device and a server.Alternatively, any method provided in the embodiments of the presentdisclosure is executed by a processor, for example, any method mentionedin the embodiments of the present disclosure is executed by theprocessor by invoking a corresponding instruction stored in a memory.Details are not described below again.

A person of ordinary skill in the art may understand that all or someoperations for implementing the foregoing method embodiments areachieved by a program by instructing related hardware; the foregoingprogram can be stored in a computer-readable storage medium; when theprogram is executed, operations including the foregoing methodembodiments are executed. Moreover, the foregoing storage mediumincludes various media capable of storing a program code such as an ROM,an RAM, a magnetic disk, or an optical disk.

FIG. 8 is a structural schematic diagram of an embodiment of a bodycontour key point detection apparatus of the present disclosure. Asshown in FIG. 8, the apparatus according to the embodiments mainlyincludes: an image feature obtaining module 800, a prediction resultobtaining module 810, and a contour key point obtaining module 820.According to one or more embodiments of the present disclosure, theapparatus may further include a neural network training apparatus 830.

The image feature obtaining module 800 is mainly configured to obtain animage feature of an image block including a body. Please refer to thedescription for operation S100 in FIG. 1 for the method embodiment abovefor the operation executed by the image feature obtaining module 800.Descriptions are not made herein repeatedly.

The prediction result obtaining module 810 is mainly configured toobtain a body contour key point prediction result of the body by meansof a first neural network according to the image feature obtained by theimage feature obtaining module 800. Please refer to the description foroperation S110 in FIG. 1 for the method embodiment above for theoperation executed by the prediction result obtaining module 810.Descriptions are not made herein repeatedly.

The contour key point obtaining module 820 is mainly configured toobtain a body contour key point in the image block according to the bodycontour key point prediction result. The body contour key point is usedfor representing an outer contour of the body. Please refer to thedescription for operation S120 in FIG. 1 for the method embodiment abovefor the operation executed by the contour key point obtaining module820. Descriptions are not made herein repeatedly.

The neural network training apparatus 830 is mainly configured to trainthe first neural network. The neural network training apparatus 830 mayalso be configured to train the input neural network and the secondneural network. The neural network training apparatus 830 may further beconfigured to mark the image sample. Please refer to the descriptionsfor FIG. 3 to FIG. 5 for the method embodiment above for the operationsexecuted by the neural network training apparatus 830. Please refer tothe description for FIG. 9 for the embodiment below for the structure ofthe neural network training apparatus 830. Descriptions are not madeherein repeatedly.

FIG. 9 is a structural schematic diagram of an embodiment of a neuralnetwork training apparatus of the present disclosure. As shown in FIG.9, the apparatus according to the embodiments mainly includes: a firstmodule 900, a second module 910, and a third module 920. According toone or more embodiments of the present disclosure, the apparatus mayfurther include: a first marking module 930 and a second marking module940.

The first module 900 is mainly configured to obtain an image feature ofa sample image block including a body.

According to one or more embodiments of the present disclosure, thefirst module 900 includes a detection sub-module, a first obtainingsub-module, and a second obtaining sub-module. The detection sub-moduleis mainly configured to perform body detection on the image sample. Thefirst obtaining sub-module is mainly configured to provide a bodydetection result and the image sample to an input neural network to betrained, to obtain a sample image block including the body with apredetermined size by means of the input neural network to be trained.The second obtaining sub-module is mainly configured to obtain the imagefeature of the sample image block using a second neural network to betrained. Please refer to the descriptions for operation S300 in FIG. 3and operations S400, S410, S420, and S430 in FIG. 4 for the methodembodiment above for the operations executed by the first module 900 andeach sub-module included therein. Descriptions are not made hereinrepeatedly.

The second module 910 is mainly configured to obtain a body contour keypoint prediction result of the body by means of a first neural networkto be trained according to the image feature. Please refer to thedescriptions for operation S310 in FIG. 3 and operation S440 in FIG. 4for the method embodiment above for the operations executed by thesecond module 910. Descriptions are not made herein repeatedly.

The third module 920 is mainly configured to use a difference betweenthe body contour key point prediction result and the body contour keypoint marking information as guidance information to perform supervisedlearning on the first neural network to be trained. The third module 920is further configured to use a difference between the body contour keypoint prediction result and the body contour key point markinginformation as guidance information to perform supervised learning onthe input neural network to be trained and the second neural network tobe trained. Please refer to the descriptions for operation S320 in FIG.3 and operation S450 in FIG. 4 for the method embodiment above for theoperations executed by the third module 920. Descriptions are not madeherein repeatedly.

The first marking module 930 is mainly configured to obtain a bodyskeleton key point of the image sample; configure an auxiliary lineaccording to the configured body contour key point marking informationin a first set and/or the body skeleton key point; and form body contourkey point marking information in a second set according to a pointselected from the auxiliary line. Please refer to the descriptions foroperations S500, S510, and S520 in FIG. 5 for the method embodimentabove for the operations executed by the first marking module 930.Descriptions are not made herein repeatedly.

The second marking module 940 is mainly configured to form body contourkey point marking information in a third set according to N1 divisionpoints on a connection line between two body contour key points in thefirst set and/or the second set; where N1 is an integer greater than 1.Please refer to the description for operation S530 in FIG. 5 for themethod embodiment above for the operation executed by the second markingmodule 940. Descriptions are not made herein repeatedly.

FIG. 10 is a structural schematic diagram of an embodiment of an imageprocessing apparatus of the present disclosure. The apparatus in FIG. 10mainly includes: a body contour key point detection apparatus 1000 and arendering module 1010.

The body contour key point detection apparatus 1000 is mainly configuredto detect the body contour key point of the image block including thebody. The body contour key point is used for representing an outercontour of the body. Please refer to the descriptions for FIGS. 1, 3, 4and 5 for the method embodiment above for the operations executed by thebody contour key point detection apparatus 1000. Descriptions are notmade herein repeatedly.

The rendering module 1010 is mainly configured to perform AR effectrendering processing on the image block according to the body contourkey point. Please refer to the descriptions for operation S610 in FIG. 6and FIG. 7 for the method embodiment above for the operations executedby the rendering module 1010. Descriptions are not made hereinrepeatedly.

Exemplary Devices

FIG. 11 illustrates an exemplary device 1100 suitable for implementingthe present disclosure. The device 1100 is a control system/electronicsystem configured in an automobile, a mobile terminal (such as a smartmobile phone), a PC (such as a desktop computer or a notebook computer),a tablet computer, or a server. In FIG. 11, the device 1100 includes oneor more processors, a communication part, and the like. The one or moreprocessors may be, for example, one or more Central Processing Units(CPUs) 1100 and/or one or more accelerating units 1113; the acceleratingunits 113 may include, but not limited to, GPU, FPGA, and other types ofdedicated processors; and the processors may perform various appropriateactions and processing according to executable instructions stored in aRead-Only Memory (ROM) 1102 or executable instructions loaded from astorage section 1108 to a Random Access Memory (RAM) 1103. Thecommunication part 1112 may include, but is not limited to, a networkcard. The network card may include, but is not limited to, an Infiniband(IB) network card. The processor may communicate with the ROM 1102and/or the RAM 1130, to execute executable instructions. The processoris connected to the communication part 1104 via a bus 1112, andcommunicates with other target devices via the communication part 1112,thereby implementing corresponding operations in the present disclosure.

Reference is made to related descriptions in the foregoing methodembodiments for the operations executed by the instructions.Descriptions are not made herein in detail. In addition, the RAM 1103may further store various programs and data required for operations ofan apparatus. The CPU 1101, the ROM 1102, and the RAM 1103 are connectedto each other via the bus 1104. In the case that the RAM 1103 exists,the ROM 1102 is an optional module. The RAM 1103 stores executableinstructions, or writes the executable instructions into the ROM 1102during running, where the executable instructions cause the CPU 1101 toexecute operations included in the foregoing segmentation method. AnInput/Output (I/O) interface 1105 is also connected to the bus 1104. Thecommunication part 1112 is integrated, or is configured to have multiplesub-modules (for example, multiple IB network cards) respectivelyconnected to the bus.

The following components are connected to the I/O interface 1105: aninput section 1106 including a keyboard, a mouse and the like; an outputsection 1107 including a Cathode-Ray Tube (CRT), a Liquid CrystalDisplay (LCD), a speaker and the like; the storage section 1108including a hard disk and the like; and a communication section 1109 ofa network interface card including an LAN card, a modem and the like.The communication section 1109 performs communication processing via anetwork such as the Internet. A drive 1110 is also connected to the I/Ointerface 1105 according to requirements. A removable medium 1111 suchas a magnetic disk, an optical disk, a magneto-optical disk, asemiconductor memory or the like is mounted on the drive 1110 accordingto requirements, so that a computer program read from the removablemedium is installed on the storage section 1108 according torequirements.

It should be particularly noted that, the architecture illustrated inFIG. 11 is merely an optional implementation mode. During specificpractice, the number and types of the components in FIG. 11 may beselected, decreased, increased, or replaced according to actualrequirements. Different functional components may be separated orintegrated or the like. For example, the accelerating unit and the CPUmay be separated, or the accelerating unit may be integrated on the CPU,and the communication part may be separated from or integrated on theCPU or the accelerating unit or the like. These alternativeimplementations all fall within the scope of protection of the presentdisclosure.

Particularly, a process described above with reference to a flowchartaccording to the embodiments of the present disclosure may beimplemented as a computer software program. For example, the embodimentsof present disclosure include a computer program product. The computerprogram product includes a computer program tangibly included in amachine-readable medium. The computer program includes a program codefor performing operations shown in the flowchart. The program code mayinclude instructions for correspondingly performing operations of themethod provided in the present disclosure.

In such implementations, the computer program is downloaded andinstalled from the network through the communication section 1109,and/or is installed from the removable medium 1111. When the computerprogram is executed by the CPU 1101, the instructions implementing thecorresponding operations recited in the present disclosure are executed.

In one or more implementations, the embodiments of the presentdisclosure further provide a computer program product configured tostore computer-readable instructions, where when the instructions areexecuted, a computer executes the body contour key point detectionmethod, the neural network training method, or the image processingmethod in any of the foregoing embodiments.

The computer program product is implemented by means of hardware,software, or a combination thereof. According to one or more embodimentsof the present disclosure, the computer program product is specificallyrepresented by a computer storage medium. In another example, thecomputer program product is represented by a software product, such asSoftware Development Kit (SDK).

In one or more implementations, the embodiments of the presentdisclosure further provide another body contour key point detectionmethod, neural network training method, and image processing method,apparatuses corresponding thereto, an electronic device, a computerstorage medium, a computer program, and a computer program product. Themethod includes: a first apparatus sends a body contour key pointdetection indication, a neural network training indication, or an imageprocessing instruction to a second apparatus, the instruction causingthe second apparatus to execute the body contour key point detectionmethod, neural network training method, or image processing method, inany one of the foregoing possible embodiments; and the first apparatusreceives a body contour key point detection result, a neural networktraining result, or an image processing result sent by the secondapparatus.

In some embodiments, the body contour key point detection indication,the neural network training indication, or the image processinginstruction may be an invocation instruction. The first apparatusinstructs, by means of invocation, the second apparatus to perform abody contour key point detection operation, a neural network trainingoperation, or an image processing operation; accordingly, in response toreception of the invocation instruction, the second apparatus mayperform operations and/or procedures in any embodiment of the foregoingbody contour key point detection method, neural network training method,or image processing method.

It should be understood that the terms such as “first” and “second” inthe embodiments of the present invention are only used fordistinguishing, and shall not be understood as limitations on theembodiments of the present invention. It should also be understood that,in the present invention, “multiple” may refer to two or more, and “atleast one” may refer to one, two or more. It should also be understoodthat, for any component, data or structure mentioned in the presentdisclosure, if there is no explicit limitation or no opposite motivationis provided in context, it is generally understood that the number ofthe component, data or structure is one or more. It should also beunderstood that, the descriptions of the embodiments in the presentdisclosure focus on differences between the embodiments, and for same orsimilar parts in the embodiments, refer to these embodiments. For thepurpose of brevity, details are not described again.

The methods, apparatuses, electronic devices, and computer-readablestorage media according to the present disclosure may be implemented inmany manners. For example, the methods, apparatuses, electronic devicesand computer-readable storage media according to the present disclosuremay be implemented by using software, hardware, firmware, or anycombination of software, hardware, and firmware. The foregoing sequenceof the operations of the method is merely for description, and unlessotherwise stated particularly, the operations of the method in thepresent disclosure are not limited to the specifically describedsequence. In addition, in some implementations, the present disclosureis also implemented as programs recorded in a recording medium. Theprograms include machine-readable instructions for implementing themethods according to the present disclosure. Therefore, the presentdisclosure further covers the recording medium storing the programs forperforming the methods according to the present disclosure.

The descriptions of the present disclosure are provided for the purposeof examples and description, and are not intended to be exhaustive orlimit the present disclosure to the disclosed form. Many modificationsand changes are obvious to a person of ordinary skill in the art. Theimplementations are selected and described to better describe aprinciple and an actual application of the present disclosure, and tomake a person of ordinary skill in the art understand the embodiments ofthe present disclosure, so as to design various implementations withvarious modifications applicable to particular use.

The invention claimed is:
 1. A body contour key point detection method,performed by an electronic device, comprising: obtaining an imagefeature of an image block comprising a body; obtaining, by means of afirst neural network, a body contour key point prediction result of thebody according to the image feature; and obtaining, according to thebody contour key point prediction result, a body contour key point inthe image block, wherein the body contour key point is used forrepresenting an outer contour of the body, wherein the first neuralnetwork is trained in advance by means of a training image setcomprising body contour key point marking information, wherein thetraining of the first neural network comprises: obtaining an imagefeature of a sample image block comprising a body; obtaining, by meansof a first neural network to be trained, a body contour key pointprediction result of the body according to the image feature of thesample image block; and using a difference between the body contour keypoint prediction result and the body contour key point markinginformation as guidance information to perform supervised learning onthe first neural network to be trained.
 2. The method according to claim1, wherein the obtaining the image feature of the image block comprisingthe body comprises: performing body detection on an image to beprocessed; and obtaining the image feature of the image block comprisingthe body in the image to be processed according to a body detectionresult; wherein the image block is the image to be processed, the imageblock is a partial image comprising a body in the image to be processed,or the image block is an image block obtained by processing the partialimage comprising the body in the image to be processed.
 3. The methodaccording to claim 2, wherein the performing the body detection on theimage to be processed comprises: performing the body detection on theimage to be processed using a body detector, to obtain a centralposition of a body bounding box and a body scale factor; wherein thecentral position of the body bounding box and the body scale factor areused for determining a position of the body bounding box in the image tobe processed.
 4. The method according to claim 1, wherein the obtaining,by means of the first neural network, the body contour key pointprediction result of the body according to the image feature comprises:forming, by means of the first neural network, at least one body contourkey point response diagram respectively corresponding to at least onebody contour key point according to the image feature.
 5. The methodaccording to claim 4, wherein each of the at least one body contour keypoint response diagram comprises a body contour key point confidencediagram, wherein the obtaining the body contour key point in the imageblock according to the body contour key point prediction resultcomprises: selecting a point meeting a predetermined confidencerequirement from any one body contour key point confidence diagramcomprised in the at least one body contour key point response diagram;and using a mapping position point of the point in the image block as abody contour key point corresponding to the any one body contour keypoint confidence diagram.
 6. The method according to claim 1, whereinthe body contour key point comprises at least one of the following: ahead contour key point, an arm contour key point, a hand contour keypoint, a shoulder contour key point, a leg contour key point, a waistcontour key point, or a foot contour key point; wherein the head contourkey point comprises at least one of: a head top key point, a nose tipkey point, or a chin key point; or the arm contour key point comprisesat least one of: a wrist contour key point, an elbow contour key point,an arm root contour key point, a lower arm contour midpoint key pointlocated at a midpoint position between the wrist contour key point andthe elbow contour key point, and an upper arm midpoint key point locatedat a midpoint position between the elbow contour key point or the armroot contour key point; or the hand contour key point comprises at leastone of: a hand tip key point and a palm midpoint key point; or theshoulder contour key point comprises at least one of: a shoulder andhead intersection key point located at an intersection position of theshoulder and the head, or a shoulder contour midpoint key point locatedat a midpoint position between the arm root contour key point and theshoulder and head intersection key point; or the leg contour key pointcomprises at least one of: a crotch key point, a knee contour key point,an ankle contour key point, a thigh root outside contour key point, ashank contour midpoint key point located at a midpoint position betweenthe knee contour key point and the ankle contour key point, a thighinner contour midpoint key point located at a midpoint position betweenthe knee inner contour key point and the crotch key point, or a thighouter contour midpoint key point located at a midpoint position betweenthe knee outer contour key point and the thigh root outside contour keypoint; or the waist contour key point comprises at least one of: N−1equal division points generated by dividing the thigh root outsidecontour key point or the arm root contour key point into N equal parts,wherein N is greater than 1; or the foot contour key point comprises atleast one of: a tiptoe key point or a heel key point.
 7. The methodaccording to claim 1, wherein the obtaining the image feature of thesample image block comprising the body comprises: obtaining an imagesample from a training data set; and obtaining an image feature of thesample image block comprising the body in the image sample.
 8. Themethod according to claim 7, wherein the obtaining, by means of thefirst neural network to be trained, the body contour key pointprediction result of the body according to the image feature comprises:forming, by means of the first neural network to be trained, at leastone body contour key point response diagram respectively correspondingto at least one body contour key point according to the image feature.9. The method according to claim 8, wherein each of the body contour keypoint response diagram comprises a body contour key point confidencediagram, wherein the using the difference between the body contour keypoint prediction result and the body contour key point markinginformation as guidance information to perform supervised learning onthe first neural network to be trained comprises: respectivelygenerating a generated body contour key point confidence diagram for atleast one piece of body contour key point marking information; and usinga difference between the body contour key point confidence diagram andthe generated body contour key point confidence diagram as guidanceinformation to perform supervised learning on the first neural networkto be trained.
 10. The method according to claim 9, wherein therespectively generating the body contour key point confidence diagramfor at least one piece of body contour key point marking informationcomprises: respectively forming a Gaussian response in a predeterminedperipheral region of a position in the sample image block correspondingto the at least one piece of body contour key point marking information,and forming a body contour key point confidence diagram separatelycorresponding to the at least one piece of body contour key pointmarking information according to the Gaussian response.
 11. The methodaccording to claim 8, wherein each of the body contour key pointresponse diagram comprises a formed body contour key point confidencediagram, wherein the using the difference between the body contour keypoint prediction result and the body contour key point markinginformation as guidance information to perform supervised learning onthe first neural network to be trained comprises: selecting a pointmeeting a predetermined confidence requirement from any one formed bodycontour key point confidence diagram comprised in the at least one bodycontour key point response diagram; using a mapping position point ofthe point in the image sample as a body contour key point correspondingto the any one formed body contour key point confidence diagram; andusing a difference between the body contour key point corresponding tothe any one formed body contour key point confidence diagram and thebody contour key point marking information of the image sample asguidance information to perform supervised learning on the first neuralnetwork to be trained.
 12. The method according to claim 7, wherein theobtaining the image feature of the sample image block comprising thebody comprises: performing body detection on the image sample; providinga body detection result and the image sample to an input neural networkto be trained, to obtain, by means of the input neural network to betrained, the sample image block comprising the body, wherein the sampleimage block has a predetermined size; and obtaining the image feature ofthe sample image block using a second neural network to be trained. 13.The method according to claim 12, wherein the training of the firstneural network further comprises: using a difference between the bodycontour key point prediction result and the body contour key pointmarking information as guidance information to perform supervisedlearning on the input neural network to be trained and the second neuralnetwork to be trained.
 14. The method according to claim 7, wherein aconfiguring process for the body contour key point marking informationcarried in the image sample comprises: obtaining a body skeleton keypoint of the image sample; configuring an auxiliary line according to atleast one of the configured body contour key point marking informationin a first set or the body skeleton key point; and forming, according toa point selected from the auxiliary line, body contour key point markinginformation in a second set.
 15. The method according to claim 14,wherein the configuring process for the body contour key point markinginformation carried in the image sample further comprises: forming bodycontour key point marking information in a third set according to N1division points on a connection line between two body contour key pointsin at least one of the first set or the second set; wherein N1 is aninteger greater than
 1. 16. The method according to claim 14, whereinthe configuring the auxiliary line according to at least one of theconfigured body contour key point marking information in the first setor the body skeleton key point comprises: determining, according to thebody skeleton key point, a first auxiliary line passing through an armroot inner contour key point and vertical to an upper arm direction; andthe forming, according to the point selected from the auxiliary line,the body contour key point marking information in the second setcomprises: forming, according to an intersection point of the firstauxiliary line and an upper arm outer contour, marking information ofthe arm root outer contour key point in the second set.
 17. The methodaccording to claim 14, wherein the configuring the auxiliary lineaccording to the configured body contour key point marking informationin the first set and/or the body skeleton key point comprises:configuring a second auxiliary line passing through a crotch key pointand vertical to a thigh direction formed by the body skeleton key point;and the forming, according to the point selected from the auxiliaryline, the body contour key point marking information in the second setcomprises: forming, according to an intersection point of the secondauxiliary line and a thigh outer contour, marking information of thethigh root outside contour key point in the second set.
 18. Anelectronic device, comprising: a processor; and a memory for storinginstructions executable by the processor; wherein execution of theinstructions by the processor causes the processor to perform: obtainingan image feature of an image block comprising a body; obtaining, bymeans of a first neural network, a body contour key point predictionresult of the body according to the image feature; and obtaining,according to the body contour key point prediction result, a bodycontour key point in the image block, wherein the body contour key pointis used for representing an outer contour of the body, wherein the firstneural network is trained in advance by means of a training image setcomprising body contour key point marking information, wherein thetraining of the first neural network comprises: obtaining an imagefeature of a sample image block comprising a body; obtaining, by meansof a first neural network to be trained, a body contour key pointprediction result of the body according to the image feature of thesample image block; and using a difference between the body contour keypoint prediction result and the body contour key point markinginformation as guidance information to perform supervised learning onthe first neural network to be trained.
 19. A non-transitorycomputer-readable storage medium, configured to store computer-readableinstructions, wherein execution of the instructions by the processorcauses the processor to perform: obtaining an image feature of an imageblock comprising a body; obtaining, by means of a first neural network,a body contour key point prediction result of the body according to theimage feature; and obtaining, according to the body contour key pointprediction result, a body contour key point in the image block, whereinthe body contour key point is used for representing an outer contour ofthe body, wherein the first neural network is trained in advance bymeans of a training image set comprising body contour key point markinginformation, wherein the training of the first neural network comprises:obtaining an image feature of a sample image block comprising a body;obtaining, by means of a first neural network to be trained, a bodycontour key point prediction result of the body according to the imagefeature of the sample image block; and using a difference between thebody contour key point prediction result and the body contour key pointmarking information as guidance information to perform supervisedlearning on the first neural network to be trained.